Method And Apparatus For Virtual Endoscopy

ABSTRACT

A surgical instrument navigation system is provided that visually simulates a virtual volumetric scene of a body cavity of a patient from a point of view of a surgical instrument residing in the cavity of the patient. The surgical instrument navigation system includes: a surgical instrument; an imaging device which is operable to capture scan data representative of an internal region of interest within a given patient; a tracking subsystem that employs electro-magnetic sensing to capture in real-time position data indicative of the position of the surgical instrument; a data processor which is operable to render a volumetric perspective image of the internal region of interest from a point of view of the surgical instrument; and a display which is operable to display the volumetric perspective image of the patient.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.15/817,914 filed on Nov. 20, 2017; which is a continuation of U.S.patent application Ser. No. 11/068,342 filed on Feb. 28, 2005, nowabandoned; which is a continuation of U.S. patent application Ser. No.10/223,847 filed on Aug. 19, 2002, now U.S. Pat. No. 6,892,090 issued onMay 10, 2005. The disclosures of the above applications are incorporatedherein by reference.

FIELD

The present teachings relates generally to surgical instrumentnavigation systems and, more particularly, to a system that visuallysimulates a virtual volumetric scene of a body cavity from a point ofview of a surgical instrument residing in a patient.

BACKGROUND

Precise imaging of portions of the anatomy is an increasingly importanttechnique in the medical and surgical fields. In order to lessen thetrauma to a patient caused by invasive surgery, techniques have beendeveloped for performing surgical procedures within the body throughsmall incisions with minimal invasion. These procedures generallyrequire the surgeon to operate on portions of the anatomy that are notdirectly visible, or can be seen only with difficulty. Furthermore, someparts of the body contain extremely complex or small structures and itis necessary to enhance the visibility of these structures to enable thesurgeon to perform more delicate procedures. In addition, planning suchprocedures required the evaluation of the location and orientation ofthese structures within the body in order to determine the optimalsurgical trajectory.

Endoscopy is one commonly employed technique for visualizing internalregions of interest within a patient. Flexible endoscopes enablesurgeons to visually inspect a region prior to or during surgery.However, flexible endoscopes are relatively expensive, limited inflexibility due to construction and obscured by blood and otherbiological materials.

Therefore, it is desirable to provide a cost effective alternativetechnique for visualizing an internal regions of interest within apatient.

SUMMARY

A surgical instrument navigation system is provided that visuallysimulates a virtual volumetric scene of a body cavity of a patient froma point of view of a surgical instrument residing in the patient. Thesurgical instrument navigation system generally includes: a surgicalinstrument, such as a guide wire or catheter; a tracking subsystem thatcaptures real-time position data indicative of the position (locationand/or orientation) of the surgical instrument; a data processor whichis operable to render a volumetric image of the internal region ofinterest from a point of view of the surgical instrument; and a displaywhich is operable to display the volumetric image of the patient. Thesurgical instrument navigation system may also include an imaging devicewhich is operable to capture 2D and/or 3D volumetric scan datarepresentative of an internal region of interest within a given patient.

For a more complete understanding of the present teachings, referencemay be made to the following specification and to the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an exemplary surgical instrument navigationsystem according to various embodiments;

FIG. 2 is a flowchart that depicts a technique for simulating a virtualvolumetric scene of a body cavity from a point of view of a surgicalinstrument positioned within the patient according to variousembodiments;

FIG. 3 is an exemplary display from the surgical instrument navigationsystem according to various embodiments;

FIG. 4 is a flowchart that depicts a technique for synchronizing thedisplay of an indicia or graphical representation of the surgicalinstrument with cardiac or respiratory cycle of the patient according tovarious embodiments; and

FIG. 5 is a flowchart that depicts a technique for generatingfour-dimensional image data that is synchronized with the patientaccording to various embodiments.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

FIG. 1 is a diagram of an exemplary surgical instrument navigationsystem 10. According to various embodiments, the surgical instrumentnavigation system 10 is operable to visually simulate a virtualvolumetric scene within the body of a patient, such as an internal bodycavity, from a point of view of a surgical instrument 12 residing in thecavity of a patient 13. To do so, the surgical instrument navigationsystem 10 is primarily comprised of a surgical instrument 12, a dataprocessor 16 having a display 18, and a tracking subsystem 20. Thesurgical instrument navigation system 10 may further include (oraccompanied by) an imaging device 14 that is operable to provide imagedata to the system.

The surgical instrument 12 is preferably a relatively inexpensive,flexible and/or steerable catheter that may be of a disposable type. Thesurgical instrument 12 is modified to include one or more trackingsensors that are detectable by the tracking subsystem 20. It is readilyunderstood that other types of surgical instruments (e.g., a guide wire,a pointer probe, a stent, a seed, an implant, an endoscope, etc.) arealso within the scope of the present teachings. It is also envisionedthat at least some of these surgical instruments may be wireless or havewireless communications links. It is also envisioned that the surgicalinstruments may encompass medical devices which are used for exploratorypurposes, testing purposes or other types of medical procedures.

Referring to FIG. 2, the imaging device 14 is used to capture volumetricscan data 32 representative of an internal region of interest within thepatient 13. The three-dimensional scan data is preferably obtained priorto surgery on the patient 13. In this case, the captured volumetric scandata may be stored in a data store associated with the data processor 16for subsequent processing. However, one skilled in the art will readilyrecognize that the principles of the present teachings may also extendto scan data acquired during surgery. It is readily understood thatvolumetric scan data may be acquired using various known medical imagingdevices 14, including but not limited to a magnetic resonance imaging(MRI) device, a computed tomography (CT) imaging device, a positronemission tomography (PET) imaging device, a 2D or 3D fluoroscopicimaging device, and 2D, 3D or 4D ultrasound imaging devices. In the caseof a two-dimensional ultrasound imaging device or other two-dimensionalimage acquisition device, a series of two-dimensional data sets may beacquired and then assembled into volumetric data as is well known in theart using a two-dimensional to three-dimensional conversion.

A dynamic reference frame 19 is attached to the patient proximate to theregion of interest within the patient 13. To the extent that the regionof interest is a vessel or a cavity within the patient, it is readilyunderstood that the dynamic reference frame 19 may be placed within thepatient 13. To determine its location, the dynamic reference frame 19 isalso modified to include tracking sensors detectable by the trackingsubsystem 20. The tracking subsystem 20 is operable to determineposition data for the dynamic reference frame 19 as further describedbelow.

The volumetric scan data is then registered as shown at 34. Registrationof the dynamic reference frame 19 generally relates information in thevolumetric scan data to the region of interest associated with thepatient. This process is referred to as registering image space topatient space. Often, the image space must also be registered to anotherimage space. Registration is accomplished through knowledge of thecoordinate vectors of at least three non-collinear points in the imagespace and the patient space.

Registration for image guided surgery can be completed by differentknown techniques. First, point-to-point registration is accomplished byidentifying points in an image space and then touching the same pointsin patient space. These points are generally anatomical landmarks thatare easily identifiable on the patient. Second, surface registrationinvolves the user's generation of a surface in patient space by eitherselecting multiple points or scanning, and then accepting the best fitto that surface in image space by iteratively calculating with the dataprocessor until a surface match is identified. Third, repeat fixationdevices entail the user repeatedly removing and replacing a device(i.e., dynamic reference frame, etc.) in known relation to the patientor image fiducials of the patient. Fourth, automatic registration byfirst attaching the dynamic reference frame to the patient prior toacquiring image data. It is envisioned that other known registrationprocedures are also within the scope of the present teachings, such asthat disclosed in U.S. Ser. No. 09/274,972, filed on Mar. 23, 1999,entitled “NAVIGATIONAL GUIDANCE VIA COMPUTER-ASSISTED FLUOROSCOPICIMAGING”, which is hereby incorporated by reference.

During surgery, the surgical instrument 12 is directed by the surgeon tothe region of interest within the patient 13. The tracking subsystem 20preferably employs electro-magnetic sensing to capture position data 37indicative of the location and/or orientation of the surgical instrument12 within the patient. The tracking subsystem 20 may be defined as alocalizing device 22 and one or more electro-magnetic sensors 24 may beintegrated into the items of interest, such as the surgical instrument12. In one embodiment, the localizing device 22 is comprised of three ormore field generators (transmitters) mounted at known locations on aplane surface and the electro-magnetic sensor (receivers) 24 is furtherdefined as a single coil of wire. The positioning of the fieldgenerators (transmitter), and the sensors (receivers) may also bereversed, such that the generators are associated with the surgicalinstrument 12 and the receivers are positioned elsewhere. Although notlimited thereto, the localizing device 22 may be affixed to anunderneath side of the operating table that supports the patient.

In operation, the field generators generate magnetic fields which aredetected by the sensor. By measuring the magnetic fields generated byeach field generator at the sensor, the location and orientation of thesensor may be computed, thereby determining position data for thesurgical instrument 12. Although not limited thereto, exemplaryelectro-magnetic tracking subsystems are further described in U.S. Pat.Nos. 5,913,820; 5,592,939; and 6,374,134 which are incorporated hereinby reference. In addition, it is envisioned that other types of positiontracking devices are also within the scope of the present teachings. Forinstance, non line-of-sight tracking subsystem 20 may be based on sonicemissions or radio frequency emissions. In another instance, a rigidsurgical instrument, such as a rigid endoscope may be tracked using aline-of-sight optical-based tracking subsystem (i.e., LED's, passivemarkers, reflective markers, etc).

Position data such as location and/or orientation data from the trackingsubsystem 20 is in turn relayed to the data processor 16. The dataprocessor 16 is adapted to receive position/orientation data from thetracking subsystem 20 and operable to render a volumetric perspectiveimage and/or a surface rendered image of the region of interest. Thevolumetric perspective and/or surface image is rendered 36 from the scandata 32 using rendering techniques well known in the art. The image datamay be further manipulated 38 based on the position/orientation data forthe surgical instrument 12 received from tracking subsystem 20.Specifically, the volumetric perspective or surface rendered image isrendered from a point of view which relates to position of the surgicalinstrument 12. For instance, at least one electro-magnetic sensor 24 maybe positioned at the tip of the surgical instrument 12, such that theimage is rendered from a leading point on the surgical instrument. Inthis way, the surgical instrument navigation system 10 according tovarious embodiments is able, for example, to visually simulate a virtualvolumetric scene of an internal cavity from the point of view of thesurgical instrument 12 residing in the cavity without the use of anendoscope. It is readily understood that tracking two or moreelectro-magnetic sensors 24 which are embedded in the surgicalinstrument 12 enables orientation of the surgical instrument 12 to bedetermined by the system 10.

As the surgical instrument 12 is moved by the surgeon within the regionof interest, its position and orientation are tracked and reported on areal-time basis by the tracking subsystem 20. The volumetric perspectiveimage may then be updated by manipulating 38 the rendered image data 36based on the position of the surgical instrument 12. The manipulatedvolumetric perspective image is displayed 40 on a display device 18associated with the data processor 16. The display 18 is preferablylocated such that it can be easily viewed by the surgeon during themedical procedure. In one embodiment, the display 18 may be furtherdefined as a heads-up display or any other appropriate display. Theimage may also be stored by data processor 16 for later playback, shouldthis be desired.

It is envisioned that the primary perspective image 38 of the region ofinterest may be supplemented by other secondary images. For instance,known image processing techniques may be employed to generate variousmulti-planar images of the region of interest. Alternatively, images maybe generated from different view points as specified by a user 39,including views from outside of the vessel or cavity or views thatenable the user to see through the walls of the vessel using differentshading or opacity. In another instance, the location data of thesurgical instrument may be saved and played back in a movie format. Itis envisioned that these various secondary images may be displayedsimultaneously with or in place of the primary perspective image.

In addition, the surgical instrument 12 may be used to generatereal-time maps corresponding to an internal path traveled by thesurgical instrument or an external boundary of an internal cavity.Real-time maps are generated by continuously recording the position ofthe instrument's localized tip and its full extent. A real-time map isgenerated by the outermost extent of the instrument's position andminimum extrapolated curvature as is known in the art. The map may becontinuously updated as the instrument is moved within the patient,thereby creating a path or a volume representing the internal boundaryof the cavity. It is envisioned that the map may be displayed in a wireframe form, as a shaded surface or other three-dimensional computerdisplay modality independent from or superimposed on the volumetricperspective image 38 of the region of interest. It is further envisionedthat the map may include data collected from a sensor embedded into thesurgical instrument, such as pressure data, temperature data orelectro-physiological data. In this case, the map may be color coded torepresent the collected data.

FIG. 3 illustrates another type of secondary image 28 which may bedisplayed in conjunction with the primary perspective image 38. In thisinstance, the primary perspective image is an interior view of an airpassage within the patient 13. The secondary image 28 is an exteriorview of the air passage which includes an indicia or graphicalrepresentation 29 that corresponds to the location of the surgicalinstrument 12 within the air passage. In FIG. 3, the indicia 29 is shownas a crosshairs. It is envisioned that other indicia may be used tosignify the location of the surgical instrument in the secondary image.As further described below, the secondary image 28 is constructed bysuperimposing the indicia 29 of the surgical instrument 12 onto themanipulated image data 38.

Referring to FIG. 4, the display of an indicia of the surgicalinstrument 12 on the secondary image may be synchronized with ananatomical function, such as the cardiac or respiratory cycle, of thepatient. In certain instances, the cardiac or respiratory cycle of thepatient may cause the surgical instrument 12 to flutter or jitter withinthe patient. For instance, a surgical instrument 12 positioned in ornear a chamber of the heart will move in relation to the patient's heartbeat. In these instance, the indicia of the surgical instrument 12 willlikewise flutter or jitter on the displayed image 40. It is envisionedthat other anatomical functions which may effect the position of thesurgical instrument 12 within the patient are also within the scope ofthe present teachings.

To eliminate the flutter of the indicia on the displayed image 40,position data for the surgical instrument 12 is acquired at a repetitivepoint within each cycle of either the cardiac cycle or the respiratorycycle of the patient. As described above, the imaging device 14 is usedto capture volumetric scan data 42 representative of an internal regionof interest within a given patient. A secondary image may then berendered 44 from the volumetric scan data by the data processor 16.

In order to synchronize the acquisition of position data for thesurgical instrument 12, the surgical instrument navigation system 10 mayfurther include a timing signal generator 26. The timing signalgenerator 26 is operable to generate and transmit a timing signal 46that correlates to at least one of (or both) the cardiac cycle or therespiratory cycle of the patient 13. For a patient having a consistentrhythmic cycle, the timing signal might be in the form of a periodicclock signal. Alternatively, the timing signal may be derived from anelectrocardiogram signal from the patient 13. One skilled in the artwill readily recognize other techniques for deriving a timing signalthat correlate to at least one of the cardiac or respiratory cycle orother anatomical cycle of the patient.

As described above, the indicia of the surgical instrument 12 tracks themovement of the surgical instrument 12 as it is moved by the surgeonwithin the patient 13. Rather than display the indicia of the surgicalinstrument 12 on a real-time basis, the display of the indicia of thesurgical instrument 12 is periodically updated 48 based on the timingsignal from the timing signal generator 26. In one exemplary embodiment,the timing generator 26 is electrically connected to the trackingsubsystem 20. The tracking subsystem 20 is in turn operable to reportposition data for the surgical instrument 12 in response to a timingsignal received from the timing signal generator 26. The position of theindicia of the surgical instrument 12 is then updated 50 on the displayof the image data. It is readily understood that other techniques forsynchronizing the display of an indicia of the surgical instrument 12based on the timing signal are within the scope of the presentteachings, thereby eliminating any flutter or jitter which may appear onthe displayed image 52. It is also envisioned that a path (or projectedpath) of the surgical instrument 12 may also be illustrated on thedisplayed image data 52.

According to various embodiments the surgical instrument navigationsystem 10 may be further adapted to display four-dimensional image datafor a region of interest as shown in FIG. 5. In this case, the imagingdevice 14 is operable to capture volumetric scan data 62 for an internalregion of interest over a period of time, such that the region ofinterest includes motion that is caused by either the cardiac cycle orthe respiratory cycle of the patient 13. A volumetric perspective viewof the region may be rendered 64 from the volumetric scan data 62 by thedata processor 16 as described above. The four-dimensional image datamay be further supplemented with other patient data, such as temperatureor blood pressure, using coloring coding techniques.

In order to synchronize the display of the volumetric perspective viewin real-time with the cardiac or respiratory cycle of the patient, thedata processor 16 is adapted to receive a timing signal from the timingsignal generator 26. As described above, the timing signal generator 26is operable to generate and transmit a timing signal that correlates toeither the cardiac cycle or the respiratory cycle of the patient 13. Inthis way, the volumetric perspective image may be synchronized 66 withthe cardiac or respiratory cycle of the patient 13. The synchronizedimage 66 is then displayed 68 on the display 18 of the system. Thefour-dimensional synchronized image may be either (or both of) theprimary image rendered from the point of view of the surgical instrumentor the secondary image depicting the indicia of the position of thesurgical instrument 12 within the patient 13. It is readily understoodthat the synchronization process is also applicable to two-dimensionalimage data acquire over time.

To enhance visualization and refine accuracy of the displayed imagedata, the surgical navigation system can use prior knowledge such as thesegmented vessel structure to compensate for error in the trackingsubsystem or for inaccuracies caused by an anatomical shift occurringsince acquisition of scan data. For instance, it is known that thesurgical instrument 12 being localized is located within a given vesseland, therefore should be displayed within the vessel. Statisticalmethods can be used to determine the most likely location within thevessel with respect to the reported location and then compensate so thedisplay accurately represents the instrument 12 within the center of thevessel. The center of the vessel can be found by segmenting the vesselsfrom the three-dimensional datasets and using commonly known imagingtechniques to define the centerline of the vessel tree. Statisticalmethods may also be used to determine if the surgical instrument 12 haspotentially punctured the vessel. This can be done by determining thereported location is too far from the centerline or the trajectory ofthe path traveled is greater than a certain angle (worse case 90degrees) with respect to the vessel. Reporting this type of trajectory(error) is very important to the clinicians. The tracking along thecenter of the vessel may also be further refined by correcting formotion of the respiratory or cardiac cycle, as described above.

The surgical instrument navigation system according to variousembodiments may also incorporate atlas maps. It is envisioned thatthree-dimensional or four-dimensional atlas maps may be registered withpatient specific scan data or generic anatomical models. Atlas maps maycontain kinematic information (e.g., heart models) that can besynchronized with four-dimensional image data, thereby supplementing thereal-time information. In addition, the kinematic information may becombined with localization information from several instruments toprovide a complete four-dimensional model of organ motion. The atlasmaps may also be used to localize bones or soft tissue which can assistin determining placement and location of implants.

While the teachings have been described according to variousembodiments, it will be understood that the teachings are capable ofmodification without departing from the spirit of the teachings as setforth in the appended claims.

What is claimed is:
 1. A navigation system to track a surgicalinstrument relative to a patient, comprising: a tracking subsystemoperable to capture in real-time position data indicative of theposition of the surgical instrument; a data processor adapted to receivescan data representative of a region of interest of a given patient andthe position data from the tracking subsystem, the data processor beingoperable to render an image of the region of interest from a point ofview which relates to the position of the surgical instrument, the imagebeing derived from the scan data; and a display in data communicationwith the data processor, the display being operable to display the imageof the patient; wherein the surgical instrument is configured to movethrough the region of interest; wherein the data processor is operableto create a map of the area through which the surgical instrument ismoved by determining the real time location and orientation of thesurgical instrument over time based on the tracking subsystem capture ofposition data indicative of the surgical instrument over time.
 2. Thenavigation system of claim 1, wherein the data processor is furtheroperable to determine in real-time the location and orientation of thesurgical instrument as the surgical instrument is moved within theregion of interest and the display is further operable to display thelocation and orientation of the surgical instrument.
 3. The navigationsystem of claim 1, wherein the map is a real-time map that is generatedand corresponds to a path traveled by the surgical instrument throughthe region of interest.
 4. The navigation system of claim 3, wherein thereal-time map is generated by an outermost extent of the trackedposition of the surgical instrument and a minimum extrapolated curvatureof the surgical instrument.
 5. The navigation system of claim 3, whereinthe display is further operable to display the real-time map; whereinthe real-time map and the image of the patient are displayedsimultaneously.
 6. The navigation system of claim 5, wherein thereal-time map is displayed superimposed on the image of the patient. 7.The navigation system of claim 5, further comprising: a sensor embeddedinto the surgical instrument; wherein the sensor is configured tocollect sensor date within the region of interest; wherein the real-timemap is color coded to represent the collected sensor data.
 8. Thenavigation system of claim 7, wherein the collected sensor data withinthe region of interest includes at least one of pressure data,temperature data, or electro-physiological data.
 9. The navigationsystem of claim 1, further comprising a timing signal generator operableto generate a timing signal, wherein: the timing signal correlates to ananatomical function; the tracking subsystem is operable to (i) receivethe timing signal from the timing signal generator, and (ii) in responseto the timing signal and the position data, update the position of thesurgical instrument to compensate for the anatomical function; the dataprocessor is operable to superimpose indicia of the surgical instrumentonto the rendered image based on the position; and the rendered image isa volumetric perspective image.
 10. The navigation system of claim 9,wherein the anatomical function is a cardiac cycle or a respiratorycycle.
 11. The navigation system of claim 10, wherein the data processoris configured to reduce flutter or jitter in the rendered image usingthe timing signal.
 12. The navigation system of claim 9, wherein thedata processor is operable to periodically update the superimposedindicia of the surgical instrument onto the rendered image based on thetiming signal from the timing signal generator.
 13. The navigationsystem of claim 1, wherein the display is operable to display a virtualvolumetric scene of an internal body cavity of the patient from a pointof view of the surgical instrument.
 14. The navigation system of claim1, wherein the map is a real-time map that corresponds to an externalboundary of an internal cavity of the patient.
 15. The navigation systemof claim 1, wherein: the data processor is configured to compensate foran inaccuracy in a tracked first position of the surgical instrumentcaused by an anatomical shift of a first anatomical member of thepatient having occurred subsequent to the acquisition of the scan data,the compensating for the inaccuracy including, determining a most likelyposition of the surgical instrument based on the tracked first position,determining whether the first position is greater than or equal to apredetermined distance away from the most likely position, andgenerating a trajectory error if the first position is greater than orequal to the predetermined distance away from the most likely position.16. A method of tracking a surgical instrument, the method comprising:receiving scan image data of a patient at a data processor; receivingposition data of a surgical instrument relative to the patient at thedata processor; rendering an image of a region of interest based on thescan image data; determining, based on the position data, the positionof the surgical instrument as the surgical instrument is moved withinthe region of interest; determine a registration with the rendered imageto determine a position of the surgical instrument relative to therendered image; generate a map of an area through which the surgicalinstrument is moved by tracking the position of the surgical instrumentover time; and displaying the map with a display device within theregion of interest and the rendered image of the region of interest. 17.The method of claim 16, further comprising displaying a virtualvolumetric scene of an internal body cavity of the patient from a pointof view of the surgical instrument.
 18. The method of claim 16, furthercomprising: receiving a timing signal that correlates to an anatomicalfunction; updating a position of the surgical instrument based upon thetiming signal to compensate for the anatomical function; and superimposeindicia of the surgical instrument onto the rendered image based on theupdated position.
 19. The method of claim 16, further comprising:capturing position data indicative of the position of the surgicalinstrument to track the position of the surgical instrument over time;and updating the map with the tracked position of the surgicalinstrument over time.
 20. The method of claim 16, further comprising:compensating for an inaccuracy in a tracked first position of thesurgical instrument caused by an anatomical shift of a first anatomicalmember of the patient having occurred subsequent to the acquisition ofthe scan data, wherein the compensating for the inaccuracy includes;determining a most likely position of the surgical instrument based onthe tracked first position; determining whether the tracked firstposition is greater than or equal to a predetermined distance away fromthe most likely position; generating a trajectory error if the trackedfirst position is greater than or equal to the predetermined distanceaway from the most likely position; and determining a second position ofthe surgical instrument based on the most likely position; anddisplaying a view relative to the determined second position.